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FOREWORD 

This  research  and  development  was  conducted  in  support  of  Navy  Decision 
Coordinating  Paper  Z0108-PN,  Education  and  Training  Development  (NDCP-Z0108-PN) , 
under  the  sponsorship  of  the  Chief  of  Naval  Operations  (Director  of  Naval 
Education  and  Training).  It  was  accomplished  under  subproject  Z0108-PN.30A 
(Adaptive  Experimental  Approach  to  Instructional  Design) . The  overall  objec- 
tive of  the  subproject  is  to  develop  an  empirically-based  Instructional  de- 
sign system  to  aid  developers  in  deciding  on  instructional  alternatives 
based  on  cost  benefits  and  resource  limitations. 

The  cooperation  and  assistance  of  personnel  at  Fleet  Aviation  Specialized 
Operational  Training  Group,  Pacific  Fleet  are  recognized  and  appreciated. 

CDR  Bruce  Churchill  made  particularly  invaluable  contributions  as  subject 
matter  expert.  The  help  of  the  computer  programmers,  Mr.  Robert  Padilla 
and  Mr.  Anthony  Sassano,  is  also  gratefully  acknowledged. 

The  results  of  this  study  are  Intended  for  use  by  the  research  and  develop- 
ment community.  In  addition,  based  upon  the  findings  and  conclusions  of  work  | 

efforts  on  low  cost  part-task  training  using  interactive  computer  graphics,  j- 

specifications  will  be  developed  and  recommended  for  incorporation  into  the  i 

planning  and  design  of  future  aircrew  training  systems  and  simulators.  1 


J.  J.  CLARKIN 
Commanding  Officer 
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SUMMARY 


Problem 

Typically,  performance  skills  training  in  the  military  is  carried  out 
on  high  fidelity  simulators.  Since  there  is  little  evidence  that  might 

suggest  a less  costly  alternative,  this  approach  has  been  used  to  ensure  ^ 

maximum  transfer  of  training.  Working  from  the  premise  that  high  fidelity 
is  not  always  necessary  for  effective  training,  the  present  research 
investigated  the  use  of  computer-based  simulations  for  low  cost  performance 
skills  training. 

Purpose 

This  effort  was  designed  as  a follow-on  to  a study  that  demonstrated  ] 

the  feasibility  of  using  graphic  simulations  presented  on  a computer-based 
training  (CBT)  system  to  teach  procedural  and  simple  perceptual  motor  skills. 

The  intention  was  to  extend  the  generality  of  the  previous  findings  by  test- 
ing the  effect  of  the  methodology  for  training  complex  perceptual  motor 
skills.  The  examination  of  the  feasibility  of  using  software  to  "link"  two 
o)'  more  computer  terminals  so  students  could  practice  simulated  Interactions 
of  operators  in  patrolling  aircraft  was  also  of  interest.  ; 

Approach  | 

CBT  materials  were  designed  to  train  students  to  operate  the  S-3A  Data  ! 

Link  11  communications  system.  Graphic  displays  of  the  functions  and  j 

appearances  of  the  necessary  controls  and  displays  were  presented  to  students 
on  the  PLATO  IV  CBT  system.  The  materials  include  segments  for  practice, 
review,  and  a d3mamic  tactical  scenario.  Instruction  and  feedback  were 
programmed  where  appropriate.  Pairs  of  students  Interacted  with  these 
materials  and  then  responded  to  a questionnaire  that  was  developed  to  assess 
student  impressions  of  this  application  of  CBT.  Expert  opinion  was  solicited  I 

from  fleet-experienced  operators  regarding  the  training  value  and  simulation 
fidelity  of  the  lesson  materials. 

Findings 

Results  showed  that  students  liked  the  use  of  computer-based  simulations 
for  training,  and  that  they  perceived  that  the  training  had  been  effective. 

It  was  also  found  that  system  communication  line  errors  were  a major  source 
of  annoyance  for  students,  and  that  these  students  were  less  tolerant  of  CBT 
system  features  than  students  in  the  previous  study.  Although  experienced 
operators  stated  that  the  training  lesson  would  be  helpful  in  learning  the 
Data  Link  operations,  they  were  critical  of  the  use  of  the  copilot  panel  and 
the  slow  display  response  times.  Cost  considerations  and  potential  implica- 
tions on  fleet  readiness  were  discussed. 

Conclusions 


It  was  concluded  that  the  use  of  inter  ctive  computer  graphics  for  simulat- 
ing two-person  mission  operations  was  a feasible  training  methodology.  This 
application  of  CBT  allowed  students  to  practice  tasks  that  could  otherwise  be 
practiced  only  in  the  operational  situation. 


This  technology  could  potentially  serve  as  a low  cost  capability  for 
shore-based  training  in  areas  where  the  alternatives  are  either  (1)  little 
or  no  training,  or  (2)  use  of  high  fidelity  simulators  or  operational  equip- 
ment. 

Recommendations 


Additional  research  and  development  should  be  performed  (1)  to  investi- 
gate the  use  of  stand-alone  terminals  for  computer-based  simulation,  (2)  to 
obtain  longitudinal  and  transfer  data,  (3)  to  examine  the  effects  of  previous 
training  and  type  of  skill  being  trained,  and  (4)  to  determine  and  investigate 
areas  of  training  for  implementation  of  new,  more  effective  technologies. 
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INTRODUCTION 


Problem 

Since  large  numbers  of  military  personnel  are  assigned  to  training 
programs  each  year,  the  Navy  has  a vested  interest  in  the  research  and 
development  of  efficient  training  technologies.  Research  at  the  Navy 
Personnel  Research  and  Development  Center  (NAVPERSRANDCEN)  has  been  directed 
toward  the  development  of  procedures  for  adapting  instruction  to  individuals 
while  maintaining  cost  effectiveness.  One  instructional  methodology 
that  has  received  much  attention  is  computer-assisted  instruction  (CAI) , 
sometimes  called  computer-based  training  (CBT) . It  has  been  shown  that 
this  methodology  provides  effective  and  efficient  training.  For  example, 
Ford,  Slough,  and  Hurlock  (1972)  reported  an  average  training  time  savings 
of  47  percent  for  CAI  as  compared  to  other  instructional  methods  that  were 
investigated.  Until  recently,  however,  the  bulk  of  CAI  or  CBT  research 
has  examined  only  the  use  of  the  computer  for  teaching  cognitive  skills, 
such  as  math.  The  successful  application  of  this  methodology  to  training 
performance  skills,  such  as  operation  of  equipment,  could  produce  con- 
siderable financial  savings  in  Navy  training  programs. 

The  fact  that  performance  skills  training  is  particularly  costly  for 
the  Navy  may  be  a result,  at  least  in  part,  of  the  state-of-the-art  in 
simulation.  These  training  programs  usually  require  students  to  have 
hands-on  practice  with  actual  equipment  or  high  fidelity  simulators  to 
learn  even  the  simplest  performance  skills.  Program  designers  generally 
prescribe  this  method  because  the  salient  transfer  of  training  variables 
is  unknown.  In  other  words,  since  the  program  designers  do  not  know 
the  minimum  fidelity  requirements  for  a training  device,  they  assume 
that  the  best  way  to  provide  instruction  is  to  use  a device  that  is 
as  similar  to  actual  equipment  as  possible.  While  numerous  researchers 
(e.g..  Cream  & Lambertson,  1975)  have  suggested  that  the  use  of  high 
fidelity  devices  is  not  always  necessary  for  training,  there  are  still 
very  few  solutions  offered  in  the  way  of  alteimate  training  methods. 

Purpose 

The  present  research  effort  was  designed  as  a follow-on  to  an  earlier 
study  (Crawford,  Hurlock,  Padilla,  & Sassano,  1976),  which  demonstrated 
the  feasibility  of  using  interactive  computer  graphic  simulations  of 
operational  equipment  for  part-task  training  of  performance  skills. 

To  extend  the  generality  of  these  findings,  the  present  research  employed 
this  methodology  to  train  S-3A  Tactical  Coordinators  (TACCOs)  to  operate 
the  Data  Link  11  system.  The  Data  Link  is  a communications  system  that 
involves  the  participation  of  operators  in  several  aircraft  interacting 
with  their  equipment  and  each  other.  Unlike  the  previous  research, 
relatively  complex  perceptual  motor  skills  are  required  to  operate  Data 
Link;  this  training  represents  an  advanced  phase  of  instruction  for  S-3A 
TACCOs. 
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There  was  a dual  interest  in  the  present  research:  (1)  to  determine 

if  computer-based  simulations  could  be  used  as  a part-task  training  device 
for  teaching  perceptual  motor  skills,  and  (2)  to  examine  the  feasibility 
of  elaborating  on  the  methodology  by  using  sof  :vare  to  "link"  two  or 
more  CBT  terminals  in  order  to  simulate  interactions  between  two  or 
more  aircraft. 

If  effective  training  is  demonstrated  in  this  project,  several  products 
are  expected.  One  benefit  would  be  to  expand  the  existent  data  base 
of  fidelity  literature  by  showing  that  perceptual  motor  skills  can  be 
taught  with  low  appearance  and  high  operational  fidelity  simulation. 

Another  would  be  the  development  of  a dynamic,  low  cost  training  capability 
for  Data  Link  in  ground  training.  Tactical  officers  seldom  have  the  oppor- 
tunity to  practice  these  skills  until  they  are  in  an  operational  S-3A 
in  the  fleet.  Finally,  it  is  expected  that,  if  computer  terminals  can 
be  "linked"  to  provide  interactive  simulations  for  two  or  more  persons, 
a new  and  potentially  beneficial  use  of  CBT  will  have  been  demonstrated. 

Background 

In  a recent  research  effort  at  NAVPERSRANDCEN,  the  training  and  cost 
effectiveness  of  teaching  procedural  and  simple  perceptual  motor  skills 
with  computer-based  simulations  were  compared  to  that  of  a conventionally 
used  method  (Crawford  et  al.,  1976).  Copilots  in  their  first  week  of 
S-3A  training  were  compared  on  their  ability  to  utilize  an  operational 
panel,  the  Integrated  Control  System,  after  training  with  one  of  the  two 
instructional  methodologies.  The  conventional  methodology  required  that 
students  study  a workbook  for  8 hours  and  then  undergo  a 1-hour  hands-on 
practice  with  a high  fidelity  position  trainer  under  the  guidance  of  an 
instructor.  The  CBT  group  practiced  for  3 hours  with  two-dimensional 
graphic  simulations  of  the  panel  and  a multipurpose  display  that  were 
presented  to  them  via  the  PLATO  IV  instructional  system.  These  materials 
were  created  by  software  that  simulated  the  appearance  and  all  functions 
of  the  controls  and  display.  The  simulations  were  presented  with  instruc- 
tion and  feedback  and  designed  to  elicit  maximum  student-system  interaction. 
Student  input  was  registered  for  computer  evaluation  when  the  student 
pointed  at  a touch-sensitive  front  panel  on  the  PLATO  IV  terminal.  Spe- 
cifically, when  the  student  touched  the  graphic  simulations,  the  components 
and  display  (including  tactical  symbology,  readouts,  alerts,  and  tableaus) 
reacted  similarly  to  actual  equipment.  Additionally,  the  computer  eval- 
uated the  input  and  gave  the  student  feedback. 

Analysis  of  the  experimental  data  showed  that  CBT  resulted  in  mastery 
of  the  requisite  skills  and  that  considerable  cost  avoidance  could  be 
achieved  by  substituting  it  for  the  conventional  method  in  this  phase 
of  S-3A  copilot  training.  Estimates  of  additional  savings  that  could 
be  obtained  if  the  CBT  system  were  utilized  on  a full  time  basis  for 
S-3A  training  made  it  worthwhile  to  pursue  this  investigation  further. 


A related  study  has  come  to  the  attention  of  the  authors  since  the 
execution  of  the  present  effort.  It  will  be  summarized  here  since  it  is 
pertinent  to  the  discussion  of  the  present  results.  Trolllp  (1977) 
demonstrated  that  students  could  learn  to  fly  holding  patterns  by  CBT. 

Like  the  present  authors,  Trollip  was  interested  in  the  Investigation  of 
CBT  for  skills  training.  Displays  of  instruments  were  presented  on  a 
PLATO  IV  instructional  system,  and  a "Joystick"  was  interfaced  with  the 
computer  for  simulated  flight.  Students  were  able  to  practice  "flying" 
holding  patterns  under  different  conditions  as  shown  on  the  simulated 
indicators.  After  each  practice,  graphic  representations  of  the  "flown" 
pattern  and  the  ideal  pattern  were  displayed  with  diagnostic  feedback. 

Students  were  trained  by  either  PLATO  or  a lecture,  and  all  were 
given  additional  training  on  a simulator  and  a performance  test.  Results 
showed  that  the  PLATO  simulations  had  provided  effective  training.  The 
author  concluded  that  complex  procedural  skills  can  be  taught  by  a computer 
when  the  necessary  cues  are  simulated. 
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METHOD 


Subjects  and  Advisors 

Twenty-two  male  S-3A  Tactical  Coordinator  (TACCO)  trainees  at  the 
Fleet  Aviation  Specialized  Operational  Training  Group  Pacific  Fleet 
(FASOTRAGRUPAC) , Naval  Air  Station,  North  Island  participated  in  the  study. 
These  men,  who  were  in  their  sixth  week  of  S-3A  training,  served  as  sub- 
jects as  part  of  their  course  work. 

Six  experienced  TACCOs  were  solicited  for  their  opinions  about  the 
PLATO  Data  Link  lesson.  Two  of  these  officers  were  TACCO  instructors  and 
familiar  with  the  PLATO  lesson.  The  others  had  just  returned  from  deploy- 
ment and  volunteered  to  go  through  the  PLATO  lesson  as  "students."  The 
background  of  the  experienced  TACCOs  Data  Link  training  consisted  of  3 
to  15  hours  of  combined  lecture/workbook  instruction.  During  fleet  deploy- 
ment, they  averaged  two  or  three  flights  per  week  and  attempted  Data  Link 
operations  during  most  flights. 

Apparatus 

Subjects  received  their  training  from  the  PLATO  IV  instructional 
system,  which  is  under  the  control  of  a CDC  Cyber  73  computer.  Data 
are  transmitted  between  the  computer  and  remote  student  terminals  by  voice 
grade  telephone  lines.  Each  terminal  has  a 64-character  keyboard  and  an 
8-1/2  inch  square  plasma  display  panel.  The  terminals  were  located  in 
individual  carrels  at  the  FASOTRAGRUPAC  building. 

The  terminals  used  in  this  research  were  additionally  equipped  with 
touch  panels  that  enabled  students  to  enter  data  into  the  computer  by 
pointing  at  any  of  the  256  programmable  areas  of  the  plasma  panel.  The 
touch  panel  consists  of  a thin  framework  that  is  substituted  for  the 
normal  metal  front  frame  of  the  PLATO  IV  terminal.  It  contains  the  neces- 
sary electronics  (scanning  system,  infrared  light  emitters  and  detectors, 
control  logic,  and  cable  to  connect  with  the  terminal  connector)  to  permit 
the  panel  to  operate  as  an  auxiliary  input  device.  The  light  emitters 
and  detectors  are  evenly-spaced  in  a 16  x 16  array.  The  invisible  paths 
formed  by  the  light  rays  across  the  exterior  surface  of  the  plasma  panel 
describe  a checkerboard-like  array  of  256  1/2-inch  square  points  that  may 
be  selected  by  the  student.  Inputs  are  sensed  by  the  system  when  the  beams 
of  light  emitting  from  the  X and  Y axes  are  broken  simultaneously  by  the 
student's  pointing  responses. 

Training  Materials 

The  materials  were  designed  to  train  students  to  use  the  S-3A  Data 
Link  11  system.  Instruction,  feedback,  definitions  of  terms,  and  functions 
and  appearances  of  necessary  controls  and  displays  were  presented  to  students 
on  the  PLATO  IV  Instructional  system.  The  PLATO  IV  computer  was  programmed 
to  present  these  materials  in  response  to  student  input  through  the  touch 
panel  or  keyboard,  and  to  evaluate  student  performance  when  appropriate. 

Pairs  of  student  terminals  were  "linked"  by  software  so  that  students  could 
simulate  tactical  interactions  as  operators  in  separate  S-3As  on  patrol. 
Detailed  descriptions  of  the  actual  Data  Link  system  and  of  each  component 
of  the  training  materials  are  presented  in  the  following  paragraphs. 
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Data  Link  System 


The  Data  Link  system  is  a computerized  communications  system  that, 
in  effect,  unites  all  S-3A  mission  patrol  units  into  a single  antisubmarine 
warfare  searching  entity.  Following  launch  from  a carrier,  the  S-3A  Picket 
Unit  (PU)  flies  to  an  assigned  operations  area  where  it  patrols  to  detect, 
track,  and  possibly  destroy  hostile  targets.  The  TACCO  in  each  patrolling 
S-3A  must  make  tactical  decisions,  based  on  the  premission  briefing  and 
the  current  tactical  situation.  He  must  decide  which  contacts,  of  the  many 
that  are  continually  appearing  on  his  Multipurpose  Display  (MPD) , to  store 
on  the  Data  Link  net.  The  TACCO  carries  out  complex  interactions  with  the 
components  of  the  Integrated  Control  System  (INCOS)  in  order  to  store  con- 
tacts on  the  net,  to  keep  them  up-to-date,  and  to  perform  associated  tactical 
operations.  Once  a contact  is  on  the  net,  it  becomes  part  of  the  whole 
tactical  picture  (as  opposed  to  a smal ’ part  corresponding  to  one  individual's 
search  area) , which  can  be  seen  on  the  displays  of  the  other  patrolling  TACCOs 
and  the  Net  Control  Station  (NCS)  on  the  carrier. 

Data  Link  operations  begin  with  the  launch  of  several  S-3As  from 
a carrier.  These  crafts  then  fly  to  an  area  to  relieve  other  patrol 
units  already  on-station.  Tne  premission  brief  provides  each  TACCO 
with  information  on  destination,  procedures,  and  any  special  instructions. 
Enroute  to  the  operations  area,  each  TACCO  initializes  the  hardware 
in  his  aircraft  as  a preparation  to  joining  the  Data  Link  net,  and 
registers  any  contacts  picked  up  on  the  way  out.  Once  on-station,  the 
TACCO  configures  the  hardware  to  receive  a data  dump  of  all  tactical 
data  acquired  by  the  unit  already  on  patrol.  Following  this  dump, 
the  relieving  unit  configures  the  hardware  again  in  order  to  join  the 
Data  Link  net  as  a participating  unit.  As  a participating  unit,  the 
TACCO  then  directs  his  aircraft  around  the  assigned  area,  picks  up 
new  contacts,  keeps  old  contacts  up-to-date,  and  follows  specified 
net  reporting  procedures. 

Throughout  the  procedures  described  above,  the  TACCO  interacts 
and  receives  feedback  from  the  NCS,  the  copilot,  the  sensor  operator, 

TACCOs  in  other  units,  and  his  equipment.  For  example,  the  computerized 
equipment  gives  him  feedback  in  the  form  of  readouts  regarding  incorrectly 
configured  hardware,  and  the  NCS  sends  out  instructions  that  need  to 
be  acknowledged  and  carried  out.  In  the  computer-based  simulation  to 
be  described  here,  these  procedures  and  interactions  were  reproduced 
to  simulate  a realistic  tactical  mission. 

The  simulation  was  conceptualized  as  a two-part  effort  for  the 
trainee:  (1)  configuration  of  hardware  and  all  operations  up  to  the  point 

at  which  the  aircraft  joins  the  net  as  a participating  unit,  and  (2)  par- 
ticipation in  a simulated  mission.  Training  behaviors  required  for  the 
first  category  are  very  straightforward  since  student  performance  must 
be  evaluated  either  as  correct  or  incorrect.  However,  few  guidelines 
currently  exist  for  evaluating  student  performance  in  the  second  category 
of  behaviors.  The  first  phase  of  the  simulation,  then,  was  considered 
as  a training  segment.  In  this  segment,  records  were  kept  by  the  computer. 
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which  gave  the  student  extensive  feedback.  While  some  feedback  was 
given  in  the  second  segment,  this  phase  was  designed  as  a practice 
vehicle  for  students  and  as  a means  by  which  the  technical  feasibility 
of  this  type  of  simulation  could  be  examined. 

Premission  Brief 

A brief  sheet  that  is  similar  to  those  used  for  actual  missions 
was  used  for  the  computer-based  simulation.  The  brief  included  such 
information  as  which  unit  to  relieve,  other  units  on  patrol,  parameters 
to  use  for  various  operations  (e.g.,  radio  frequency  for  the  data  dump), 
and  all  net  reporting  procedures. 

Introduction  and  Glossary 

An  on-line  introduction  was  presented  to  give  the  student  an  orien- 
tation to  the  training  objectives  of  the  simulation  materials.  Information 
was  also  provided  on  the  use  of  PLATO  IV.  The  student  was  given  practice 
using  the  touch  panel,  and  instructed  on  how  to  use  the  keyboard  to  enter 
or  exit  at  any  point  in  the  lesson. 

A glossary  was  always  available  for  the  student's  use.  It  contained 
terms  that  the  student  had  been  previously  exposed  to,  but  that  might  be 
useful  for  review.  The  glossary  was  programmed  so  that  the  student  could 
select  a term  and  the  definition  of  it  would  be  presented  on  his  display. 

Integrated  Control  System  and  Multipurpose  Display 

The  actual  Integrated  Control  System  (INCOS))  panel  consists  of 
a series  of  rotating  matrices  of  system  option  switches,  display  scale 
switches,  numeric  switches,  and  a trackball.  The  computer-based  simulation 
of  the  panel  was  an  accurate  two-dimensional  representation  including  the 
capability  to  display  324  different  switch  labels  that  changed  as  a result 
of  student  touch  panel  input  (see  Figure  1).  Programmed  logics  also 
enabled  the  color  patterns  of  the  switches  to  change  when  appropriate  to 
indicate  various  options  (i.e.,  "not  available,"  "available,"  or  "active"). 
The  trackball  caused  a cursor  to  move  across  the  Multipurpose  Display  (^!PD) 
when  touched  repeatedly  on  the  area  which  corresponded  to  the  desired 
direction  of  movement  of  the  cursor. 

The  MPD,  which  was  always  presented  on  the  PLATO  screen  with  the 
IN'COS  panel,  was  the  area  in  wliich  all  symbology  and  tactical  actions  were 
displayed.  Using  the  panel,  the  student  could  create  svanbology,  see  all 
air  and  surface  crafts  moving,  measure  distances,  and  cause  numerous  other 
events  to  occur  that  would  all  be  seen  on  the  display,  as  in  an  operational 
S-3A. 
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Figure  1.  Computer-based  simulation  of  the 

Integrated  Control  System  (INCOS)  panel. 
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The  panel  and  MPD  were  available  in  two  separate  sections  of  the 
simulation  materials.  In  one  section,  this  equipment  was  presented  for  non- 
structured  practice.  For  example,  the  student  could  practice  configuring 
hardware  for  some  phase  of  operations  and  would  get  dynamic  reactions 
from  the  panel  and  MPD  but  without  feedback.  In  the  second  section,  the 
simulated  equipment  was  available  for  use  in  the  scenario.  At  this  point, 
the  student  would  freely  interact  with  the  equipment  while  the  programmed 
scenario  was  causing  symbology  to  move  and  messages  and  alerts  to  come 
from  the  Net  Control  Station  (NCS)  and  other  sources. 

Scenario 

The  scenario  began  with  instructions  to  the  student  tc,  initialize 
his  hardware  to  join  the  Data  Link  net  and  to  configure  to  receive  a 
data  dump  from  the  Picket  Unit  (PU)  on-station.  These  procedures  required 
simulated  interactions  with  equipment  and  verbalizations  with  the  NCS. 

When  tasks  were  completed,  the  student's  MPD  became  filled  with  the  tactical 
data  picked  up  by  the  on-station  unit.  Students  at  separate  PLATO  terminals 
participated  in  the  scenario  in  pairs,  so  each  student  was  receiving 
similar  information  concurrently.  The  total  simulated  mission  area  was 
2048  X 2048  miles.  This  was  divided  into  four  possible  patrol  areas 
of  1024  X 1024  miles  each.  All  tactical  symbology  conformed  to  this 
scale  such  that  when  a student  Increased  his  display  scale  to  see  the 
area  beyond  his,  the  display  reflected  the  symbology  in  another  PU's 
area.  Once  two  students  had  successfully  joined  the  net,  they  could 
interact  with  each  other  and  see  the  results  of  these  interactions  on 
their  MPDs. 

As  PUs  on  the  simulated  mission,  the  students  would  begin  by  dropping 
f ly-to-points  to  steer  their  aircrafts  in  some  logical  pattern  (in  this 
case,  the  brief  instructions  were  to  fly  a "ladder"  pattern)  around  the 
operations  area  in  search  of  contacts.  Eighteen  contacts  were  programmed 
to  be  "hidden"  in  each  student's  operations  area.  Each  contact  would 
be  picked  up  as  forward  looking  infrared  (FLIR) , radar,  or  visual.  When 
the  student  had  his  aircraft  within  25  miles  of  a contact,  he  would  see 
it  on  his  screen  as  a radar  fix.  The  contact  appeared  as  either  (1)  friendly, 
unknown,  or  hostile,  and  (2)  surface,  subsurface,  or  air.  The  student  could 
then  "fly"  his  craft  closer  to  the  contact;  once  he  was  within  12  miles, 
the  contact  would  show  up  as  a FLIR  fix.  To  get  a visual  fix  on  a contact, 
the  student  had  to  have  his  aircraft  within  2 miles  of  it.  The  premission 
briefing  had  instructed  the  student  to  report  any  unknown  or  hostile 
contacts  to  the  Data  Link  net,  and  to  cease  report  on  contacts  that  were 
classified  as  friendly.  The  computer  recorded  the  student's  net  reporting 
procedures,  and  his  responses  to  alerts  and  orders.  Minimal  feedback 
was  provided  for  the  student  in  this  section. 
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Student  Questionnaire 

Since  a ground  capability  for  Data  Link  training  does  not  exist, 
a questionnaire  was  developed  to  determine  student  impressions  of  the 
CBT  experience.  Items  on  the  questionnaire  were  specifically  designed 
Co  elicit  responses  about  the  PLATO  IV  system,  the  Data  Link  training 
materials,  and  CBT  as  a mode  of  instruction.  Student  feelings  regarding 
acceptability  and  perceived  effectiveness  of  the  training  were  of  major 
interest . 

Procedure 

All  students  (N  = 22)  practiced  with  the  simulation  materials  on  PLATO  IV 
for  2 hours.  As  pairs  of  students  reported  for  training,  each  was  asked 
to  sit  at  a PLATO  IV  terminal  and  read  the  premission  brief  sheet.  Each 
student  signed  onto  PLATO  IV  and  began  to  proceed  through  the  materials. 

The  order  in  which  most  of  the  materials  were  presented  was  determined  by 
student  control,  although  it  was  necessary  for  two  students  to  interact  with 
the  scenario  segment  at  the  same  time.  Following  PLATO  IV  training,  all 
students  responded  to  the  questionnaire.  The  experienced  TACCOs  responded 
to  a short  questionnaire  and  were  Interviewed  by  the  experimenters  to  obtain 
their  opinions  and  reactions  regarding  the  PLATO  lesson. 

I 
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RESULTS 


All  responses  to  the  student  questionnaire  are  presented  in  the  Appendix 
and  sunnnarized  in  the  following  paragraphs.  Open-ended  questions  were 
content-analyzed  by  two  raters  and  an  inter-rater  reliability  measure 
of  85  percent  was  obtained. 

PLATO  IV  Ratings 

Questions  1 through  6 asked  students  to  rate  the  PLATO  system  on  opera- 
tional characteristics  such  as  ease  of  "signing-on,"  readability  of  the 
display,  etc.  As  shown  in  Table  1,  three-fourths  of  the  students  rated  the 
system  favorable. 


Table  1 


Mean  Percentages  of  Responses  to 
PLATO  IV  Questions 
(N  = 22) 


Rating 

Mean  Percent 

Very  Favorable 

22 

Favorable 

53 

Neutral 

19 

Unfavorable 

4 

Very  unfavorable 

2 

Training  Materials  Ratings 

Questions  7 through  14  asked  students  to  rate  the  training  materials. 
Questions  on  the  Data  Link  system  itself  asked  students  to  Indicate:  (1) 

the  extent  to  which  it  had  helped  their  understanding  of  the  basic  skills, 
(2)  why  they  felt  it  had  or  had  not  been  effective,  and  (3)  how  they  thought 
it  would  affect  their  performance  in  the  fleet  (Nos.  7,  8,  and  11,  respec- 
tively. Responses  showed  that  this  new  training  mode  was  perceived  as  help- 
ful to  a "great  extent"  by  50  percent  of  the  students,  and  to  "some  extent" 
by  41  percent.  Most  students  (84%)  felt  that  it  was  effective,  primarily 
because  it  allowed  hands-on  practice.  The  reasons  why  students  felt  the 
courseware  was  (or  was  not)  effective  are  shown  in  Table  2. 
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Table  2 


Reasons  Given  as  to  Wliy  Training  Was  (or  Was  Not)  Effective 


Student  Comment 

% Student  Response 

No  response 

.04 

It  was  not  effective  because  it  lacked  realism 

.12 

It  was  effective  - miscellaneous  reason 

It  was  effective  because  it  allowed  visualization  of 

.20 

operations 

.24 

It  was  effective  because  it  allowed  hands-on  practice 

.40 

Regarding  tlie  effect  of  the  training  on  their  later  performance,  most 
(86%)  of  the  students  reported  that  the  experience  would  make  it  easier 
to  learn  the  skills  when  using  actual  S-3A  equipment;  and  lA  percent, 
that  it  would  enable  them  to  operate  the  Data  Link  system. 

Students  were  asked  how  they  felt  about  the  effectiveness  of  interactive 
graphic  simulations  and  feedback  in  training  (questions  9 and  10).  As  shown 
in  Table  3,  all  of  the  students  rated  the  graphic  simulations  as  either  "very 
favorable"  or  "favorable,"  compared  to  72  percent  for  feedback. 


Table  3 

Percentage  of  Student  Ratings  Assigned  to  Two  Training  Features 


Rating 

Interactive 

Graphic 

Simulations 

Feedback 

Very  favorable 

.09 

.09 

Favorable 

.91 

.63 

Neutral 

.00 

.23 

Unfavorable 

.00 

.05 

Very  unfavorable 

.00 

.00 

Student  responses  to  questions  concerning  what  they  liked  best  and 
least  about  the  training  materials  are  given  in  detail  in  the  Appendix 
under  questions  12  and  13.  In  sumnuiry,  the  most-liked  features  were  the 
hands-on  practice  capability  (38%)  and  the  fact  that  they  were  able  to 
see  a representation  of  the  equipment  operations  (19%).  The  least-liked 
features  were  system  or  software  problems  (38%)  and  delays  or  slowness 
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in  the  construction  of  graphic  displays  (23%).  (Software  or  system  problems, 
here  and  in  other  questions,  refers  to  system  interruptions,  transmission 
errors,  student  backouts  caused  by  a heavy  user  load,  and  problems  with 
the  touch  panel  caused  by  the  small  size  of  the  touch  points.) 

In  response  to  the  question  asking  for  their  opinion  on  how  the  train- 
ing could  be  improved  (No.  14),  36  percent  suggested  eliminating  system 
or  software  problems.  No  other  common  answers  were  given  to  this  question. 

CBT  Ratings 

Questions  15  through  19  asked  the  students  to  rate  CBT.  In  reporting 
attitudes  about  CBT  as  compared  to  other  types  of  training  (Nos.  15  and 
16),  the  majority  of  the  students  rated  it  either  as  "enjoyable"  (67%) 
or  "very  enjoyable"  (28%),  and  as  "outstanding"  (67%)  or  "above  average" 

(19%). 

Some  of  the  characteristics  students  reported  they  liked  most  about 
CBT  (No.  17)  were  self-paced  instruction  (29%),  Instructional  characteristics 
such  as  feedback  (21%),  the  novelty  (14%),  and  the  capability  for  hands-on 
practice  (14%).  The  least-liked  features  of  CBT  (No.  18)  reported  were  that 
it  was  tiring  or  boring  (36%)  and  had  system  or  software  problems  (23%). 

Question  19  asked  students  to  make  a judgment  as  to  how  much  value 
the  training  would  have  for  new  inexperienced  students.  Over  half  (59%) 
said  it  would  have  "much  value";  and  27  percent,  "moderate  value." 

The  last  question  (No.  20)  asked  the  students  for  any  final  comments. 

Most  (73%)  students  did  not  respond;  27  percent  made  miscellaneous  comments 
that  generally  referred  to  their  entire  training  experience  rather  than 
just  the  PLATO  segment. 

Expert  Evaluation 

The  six  experienced  TACCOs  were  asked  to  compare  the  usefulness  of 
lecture/workbook  Instruction  to  PLATO  training  for  helping  a new  TACCO 
to  operate  Data  Link  during  fleet  deployment.  On  a scale  of  0 to  10, 
where  0 = No  Help,  5 = Moderately  Helpful,  and  10  = Extremely  Helpful, 
they  gave  lecture/workbook  Instruction  an  average  rating  of  4.7;  and  PLATO, 

6.7.  Only  one  TACCO  rated  PLATO  below  6,  while  half  of  the  TACCOs  rated 
the  lecture/workbook  instruction  2 or  below. 

The  most  frequently  (N  = 3)  mentioned  criticism  of  the  PLATO  lesson 
and  scenario  was  that  the  graphic  display  rewrite  and  keyboard  response 
times  were  slower  than  real  time.  The  other  most  common  criticism  (N  = 3) 
was  that  the  scenario  was  too  complex,  having  too  many  targets  and  sightings 
displayed  at  the  same  time. 

Advantages  of  the  PLATO  training  most  often  listed.  Included:  (1) 

active  participation  by  the  trainee  (N  = 3),  (2)  extensive  help  and  instruction 
available  (N  “ 3),  (3)  self-paced  instruction  (N  = 2),  and  (4)  capability 
for  students  to  do  "dump"  operation  (N  “ 2). 
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Suggestions  for  Improving  the  PLATO  lessons  included  (1)  using  the 
TACCO  hardware  configurations  for  the  graphic  displays,  instead  of  the 
copilot  INCOS  panel,  and  (2)  placing  less  information  in  the  scenario 
displays. 
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DISCUSSION 

The  use  of  Interactive  computer  graphics  for  simulating  two-person 
mission  operations  was  shown  to  be  a feasible  methodology  for  part-task 
training.  Students  were  able  to  participate  in  simulated  operations  of 
TACCOs  on  patrol  through  computer  terminals  "linked"  by  software.  Overall 
indications  from  students  were  that  the  methodology  was  effective  for 
training  the  perceptual  motor  skills  required  for  Data  Link  operations. 

This  successful  application  demonstrates  the  feasibility  of  CBT  as  a 
potential  partial  alternative  to  costly  high  fidelity  simulators.  Student 
impressions  and  cost  considerations  will  be  discussed  in  the  following 
paragraphs. 

Student  Acceptance 

Student  attitudes  were  quite  favorable  towards  the  Data  Link  training 
experience.  They  liked  the  PLATO  system,  the  training  materials,  and  the 
mode  of  instruction.  Specifically,  students  made  positive  comments  about 
the  interactive  computer  graphics.  They  said  that  these  helped  the 
training  process  by  allowing  hands-on  practice  and/or  visualization  of 
equipment  and  mission  operations.  They  also  liked  the  self-paced  instruc- 
tion and  feedback. 

Despite  students'  overall  positive  attitude  toward  the  training,  one 
problem  that  elicited  negative  comments  (during  training  as  well  as  on 
the  questionnaire)  concerned  system  interruptions.  These  interruptions 
were  usually  caused  by  transmission  errors  on  the  phone  line  that  connects 
the  terminals  in  San  Diego  to  the  computer  at  the  University  of  Illinois. 
This  type  of  error  distorts  the  graphic  display  and  requires  the  generation 
of  a new  display.  This,  in  turn,  often  meant  that  the  student  had  to 
wait  for  the  new  display  and  then  repeat  a portion  of  the  training  segment 
that  had  previously  been  near  completion.  It  was  observed  that  this 
frustrated  students  and  caused  a severe  disruption  in  the  continuity  of 
the  learning  process.  Whether  student  acceptance  of  these  circumstances 
could  be  sustained  over  an  extended  period  of  time  is  questionable. 

The  problem  of  communication  line  errors  should  be  considered  in  future 
planning.  One  solution  would  be  the  use  of  stand-alone  terminals  because 
they  require  no  communication  line  hook-ups. 

Students  also  indicated  some  problems  with  readability  of  the  graphic 
displays.  To  display  the  entire  INCOS  panel  on  one  screen,  it  was  necessary 
to  use  small  characters  for  the  switch  labels  (see  Figure  1).  This  suggests 
that,  when  designing  graphic  simulations,  considerable  attention  should  be 
given  to  determining  the  size  of  the  alphanumer ics  that  will  ensure 
readability.  Certain  types  of  equipment,  due  to  size  or  detail,  may  have 
to  be  simulated  in  portions  to  overcome  this  type  of  problem. 

It  is  of  Interest  to  note  that,  when  the  same  simulated  panel  was 
used  in  previous  research  (Crawford  et  al.,  1976),  the  students  had  fewer 
complaints  than  did  those  in  the  current  study.  The  basic  differences 
between  the  two  studies  are  summarized  in  Table  A. 
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Table 


Because  the  training  effectiveness  was  tested  and  evaluated  to  have 
been  high  in  both  studies,  fidelity  of  the  simulation  per  se  does  not 
seem  to  offer  a logical  explanation  for  the  more  critical  attitude  of 
the  TACCOs.  A more  plausible  interpretation  may  be  that  the  TACCOs  were 
less  tolerant  of  fidelity  limitations  and  CBT  system  features,  such  as 
line  errors,  because  of  their  background  with  the  high  fidelity  position 
trainer.  Another  possibility  may  be  an  effect  caused  by  the  interaction 
of  background  with  the  advanced  nature  of  the  skills  being  trained.  For 
example,  a student  who  has  spent  considerable  time  on  the  position  trainer 
may  be  more  i'^oatient  with  fidelity  limitations  when  he  is  being  required  to 
learn  complex  skills  than  when  he  is  learning  more  simple  tasks. 

Training  Effectiveness 

Students  perceived  that  the  training  was  helpful  for  learning  the 
prerequisite  skills,  and  that  it  would  enhance  their  performance  in  the 
operational  situation.  These  results,  in  conjunction  with  those  discussed 
earlier  (Crawford  et  al.,  1976;  Trollip,  1977),  indicate  some  of  the 
possible  factors  responsible  for  the  effective  application  of  this 
methodology  for  part-task  training. 

The  use  of  computer-based  simulations  may  represent  the  best  of  both 
worlds  of  simulation  and  CBT.  The  methodology  permits  the  student  to  have 
hands-on  interactions  with  visual  representations  of  operational  equipment 
while  concurrently  receiving  instruction  and  corrective  feedback.  Any 
instructional  features,  such  as  self-paced  learning  materials,  may  also  be 
incorporated  into  these  simulations.  CBT  usually  requires  less  student 
learning  time  than  conventional  methodologies,  and  the  need  for  an  instructor 
is  eliminated  since  the  computer  can  record  and  evaluate  student  responses 
as  well  as  present  instruction  and  feedback.  Thus,  the  dynamic  features 
of  simulation  that  permit  the  student  to  actively  respond  are  combined 
with  the  efficient  and  Instructionally  beneficial  characteristics  of  CBT. 

The  present  authors  make  the  assumption  that,  for  most  military  training, 
the  appropriate  emphasis  in  the  design  of  a training  device  should  be  on 
the  manner  in  which  that  device  is  to  be  used  rather  than  on  its  physical 
similarity  to  actual  equipment.  Specifically,  active  responding,  feedback, 
self-pacing,  and  other  established  learning  principles  are  considered  to 
be  crucial  to  the  success  of  the  learning  process.  This  assumption  may 
break  down  at  the  level  at  which  gross  motor  skills  are  to  be  trained. 

However,  teaching  this  type  of  skill  is  not  usually  the  case  in  military 
training.  Rather  than  learning  to  press  a button,  the  student  is  more 
often  learning  some  cognitive  process  that  will  lead  him  to  press  a button 
in  order  to  achieve  some  desired  result. 

Trollip  (1977)  also  attributes  much  Importance  to  the  design  of  the 
instructional  materials  in  his  simulations.  He  states  that  his  experimental 
results  support  the  idea  that,  for  some  kinds  of  training,  simulation  of 
the  necessary  cue,  rather  than  simulation  of  a physical  environment,  may 
be  advantageous.  He  also  emphasizes  the  Importance  of  the  role  of  part-task 
training  as  contributing  to  effective  learning  with  computer-based  simulations. 
He  notes  that  the  student  can  practice  the  skills  for  the  particular  task 
being  trained  without  distraction  from  irrelevant  stimuli. 
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Expert  Evaluation 


The  evaluation  of  the  PLATO  lesson  by  experienced  TACCOs  was  almost 
identical  to  that  given  by  TACCO  students.  The  "experts"  felt  that  the  PLATO 
lesson  was  useful  for  preparing  students  for  fleet  deployment  and  operation 
of  the  Data  Link. 

Their  criticisms  and  suggestions  for  improving  the  PLATO  lesson  further 
support  the  conclusion  that,  as  experience  with  actual  equipment  increases, 
student  and  instructor  tolerance  and  acceptance  of  lower  fidelity  training 
devices  decreases.  This  appears  to  be  true  independent  of  training  effective- 
ness of  the  instructional  method.  For  instance,  all  but  one  of  the  experienced 
TACCOs  complained  that  the  copilot  INCOS  keyboard  was  used  instead  of  the 
TACCO  keyboard.  On  the  other  hand,  no  student  complained  about  this. 

Cost  Considerations 


Previous  research  (Crawford  et  al.,  1976)  reported  that  a cost  avoidance 
of  $44,000  per  year  could  be  accrued  by  the  S-3A  training  community  if 
PLATO  IV  terminals  and  training  materials  developed  by  NAVPERSRANDCEN  were 
operationally  implemented  in  place  of  the  conventional  training  program. 

Since  the  materials  represented  only  an  8 percent  utilization  of  time 
available  on  the  PLATO  terminals,  it  was  further  predicted  that  a $500,000 
yearly  cost  avoidance  would  be  possible  if  maximum  terminal  utilization  was 
achieved.  This  assumed  that  additional  courseware  would  be  comparable  in 
cost  effectiveness.  Since  the  present  effort  represents  new  PLATO  training 
for  the  same  group,  it  is  of  interest  to  bring  the  cost  figures  up  to  date. 

FASOTRAGRUPAC  has  continued  to  use  the  previously  developed  INCOS  copilot 
materials  for  experimental  training  while  the  PLATO  terminals  were  still 
on  the  site  for  the  Data  Link  research.  In  fact,  they  have  increased  this 
usage  by  2 hours  to  provide  a "review"  for  the  men  returning  from  10  weeks 
of  training  in  other  operations.  This  review  previously  consisted  of  4 
hours  of  workbook  study  and  1 hour  of  instructor-guided  exercises  in  the 
position  trainer.  The  Increased  usage  changes  the  potential  cost  avoidance 
for  copilot  S-3A  training. 

At  this  time,  there  are  new  figures  available  for  estimating  PLATO 
costs.  A four-terminal  configuration  currently  costs  $47,784  for  1 year. 

This  total  includes  a 10  year  amortization  of  the  terminal  purchase  cost 
($4,200),  computer  time  sharing  ($37,400),  and  terminal  maintenance  ($6,144). 

If  the  total  yearly  cost  is  divided  by  the  total  number  of  hours  the  four 
terminals  can  be  used  (8000),  the  resulting  delivery  cost  is  about  $6  per  hour. 

A new  figure  for  potential  cost  avoidance,  which  would  Include  the 
additional  2 hours  utilized  for  review  purposes,  requires  examination  of 
all  the  costs  involved  in  both  the  PLATO  training  and  the  conventional 
program  that  it  would  replace.  These  costs  are  presented  in  Table  5,  which 
shows  that  the  cost  difference  is  $345  per  student.  Thus,  for  200  students 
trained  per  year,  the  total  cost  avoidance  would  be  $69,000.  This  training 
represents  a 45  percent  utilization  of  available  terminal  time. 
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Table  5 

Cost  Per  Student  of  Conventional  and  PLATO 
Methodologies  for  One  Training  Program 


Item  Cost  ($)  Total  ($) 


Conventional  Training 
Personnel  Costs 


12  hours  student  workbook  study  0$29  per  hour  348 

2 hours  student  time  in  position  trainer  exercise  58 
2 hours  Instructor  time  in  position  trainer 

exercise  @$29  per  hour  58 

Position  Trainer  Costs 

2 hours  @$28  per  hour  56 

TOTAL  520 


PLATO  Training 
Personnel  Costs 

5 hours  student  time  145 

PLATO  Costs 

5 hours  @$6  per  hour  30 

175 


TOTAL 


345 


The  nature  of  Data  Link  training  inhibits  cost  comparison  since  students 
do  not  usually  have  the  chance  to  practice  these  skills  prior  to  deployment. 
On  occasion,  students  were  able  to  sit  in  a stationary  S-3A  and  practice 
linking  with  simulated  equipment  in  a local  Navy  lab.  The  cost  of  opera- 
tionalizing the  necessary  systems  in  the  stationary  aircraft  is  $160  per 
hour.  This  comparison  is  inadequate,  however,  since  the  rare  availability 
of  time  and  functioning  hardware  made  the  probability  of  students  receiving 
this  training  very  low.  From  the  standpoint  that  existing  copilot  INCOS 
training  materials  on  the  PLATO  system  already  hold  the  potential  for  yield- 
ing $69,000  per  year  cost  avoidance,  there  would  be  no  additional  cost  in- 
curred with  the  implementation  of  the  Data  Link  materials. 

Osage  of  the  Data  Link  training  program  brings  current  terminal 
utilization  to  57  percent.  While  it  is  possible  that  maximum  terminal 
utilization  could  produce  the  kind  of  significant  savings  that  have  been 
mentioned  above,  it  will  probably  have  to  be  realized  in  new  training 
programs.  To  talk  of  implementation  of  new  methodologies  in  an  on-going 
program,  such  as  S-3A  training,  is  an  oversimplification  of  the  issues. 

For  example,  this  does  not  consider  the  dollars  already  spent  on  high 
fidelity  simulators  that  were  determined  as  necessary  in  the  original 
analysis  of  S-3A  training  requirements.  The  magnitude  of  the  costs  involved 
prohibits  major  changes  at  this  point  in  time.  Thus,  research  that  shows 
where  high  fidelity  simulators  can  be  replaced  by  less  expensive  devices 
will  have  a more  significant  Impact  in  the  design  of  future  training  pro- 
grams . 

Additional  Training  Benefits 

A final  note  regarding  cost  considerations  is  the  recent  development 
of  low  cost,  stand-alone,  intelligent  terminals.  Where  only  a few  terminals 
are  required  for  a training-  application,  delivery  of  CBT  by  this  method 
may  be  much  more  cost  effective  than  leasing  computer  time  for  a vendor's 
large  network  system.  All  alternative  systems  should  be  compared  when 
making  an  implementation  decision. 

It  is  Important  to  examine  some  of  the  less  tangible  features  of  the 
training  described  here — features  that  relate  to  increased  effectiveness 
in  present  as  well  as  in  future  training.  If  these  characteristics  can  lead 
to  improved  fleet  readiness,  and  there  is  reason  to  think  they  can,  then 
they  should  be  given  primary  consideration.  The  basic  issue  is  the  impor- 
tance of  shore-based  vs.  fleet  training.  It  seems  reasonable  to  assume 
that  students  who  have  had  the  chance  to  practice  required  operations 
on  the  ground  at  school  may  be  better  trained  than  those  who  have  had  to 
learn  while  airborne  in  the  fleet.  Once  a man  is  in  the  air,  it  is 
unlikely  that  he  will  have  time  to  practice  a part  of  his  job  (e.g.. 

Data  Link  operations)  Independently  of  his  other  tasks.  He  is  then 
subject  to  distractions,  pressures,  and  the  lack  of  corrective  feedback, 
which  he  would  not  experience  in  shore-based  training. 
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Correct  operation  of  the  Data  Link  system,  like  other  jobs  that  are 
active  defense  functions,  is  crucial  to  the  success  of  the  mission. 

There  are  probably  many  other  similar  training  areas  that  could  benefit 
from  practice  before  deployment.  With  the  increasing  sophistication  of 
technologies  such  as  computer-based  simulation,  these  training  needs  should 
not  be  ignored.  The  research  community  should  continue  to  take  responsibility 
to  research,  design,  and  help  Implement  technological  developments  that 
could  result  in  Improved  efficiency  in  a multitude  of  Navy  jobs. 
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CONCLUSIONS 


Students  liked  the  CBT  and  felt  that  it  was  effective  in  teaching  them 
the  requisite  skills  for  Data  Link  operations.  This  particular  application 
of  CBT  allowed  students  to  practice  tasks  that  would  otherwise  be  carried 
out  in  the  operational  situation.  Thus,  a ground  training  capability  has 
been  developed  where  none  existed,  and,  in  doing  so,  the  feasibility  of  a 
unique  application  of  CBT  was  further  established.  While  additional  data 
is  desirable  (e.g.,  transfer  data),  the  potential  effectiveness  and  efficiency 
of  this  kind  of  low  appearance  fidelity  methodology  has  been  demonstrated. 

The  cost  benefits  to  be  accrued,  particularly  the  potential  training 
benefits  discussed,  suggest  that  it  Will  be  advantageous  to  do  further 
research.  It  will  be  necessary  to  develop  criteria  to  identify  those  train- 
ing situations  that  will  most  benefit  from  the  implementation  of  computer- 
based  simulations  for  training  performance  skills. 
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RECOMMENDATIONS 


Recommendations  for  future  research  are  made  with  three  goals  in 
mind:  (1)  the  advancement  of  a promising  technology,  (2)  the  development 

of  capabilities  to  retrofit  on-going  training  programs,  and  (3)  the  develop- 
ment of  specifications  to  Incorporate  these  capabilities  into  future  train- 
ing programs. 

The  following  recommendations  are  made; 

1.  Computer-based  simulations  should  be  programmed  on  stand-alone 
terminals  and  evaluated.  This  would  eliminate  the  major  problem  of  system 
interruptions  and  provide  data  for  a cost  analysis  that  could  be  compared 
to  the  figures  obtained  for  use  of  a network  system  such  as  PLATO  IV. 

2.  Data  should  be  obtained  that  show  transfer  effects  from  computer- 
based  training  to  the  operational  situation.  It  has  been  shown  that  low 
fidelity  simulations  are  less  expensive  and  provide  training  as  effective 
as  that  carried  out  on  actual  equipment.  However,  it  is  not  known  whether 
or  not  there  is  any  degradation  in  performance  over  time. 

3.  The  present  research  indicated  the  need  to  examine  the  effects  of 
student  sophistication  (l.e.,  amount  of  previous  training)  and  type  of 
skill  being  trained  as  these  relate  to  student  acceptability,  fidelity  of 
training  device,  and  training  effectiveness  of  computer-based  simulations. 

4.  Empirical  comparisons  of  shore-based  and  fleet  training,  in  terms 
of  effectiveness,  are  necessary.  These  could  lead  to  firm  recommendations 
for  Implementation  of  training  technologies  such  as  the  one  described  here. 
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APPENDIX 

RESPONSES  TO  STUDENT  QUESTIONNAIRE 
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RESPONSES  TO  STUDENT  QUESTIONNAIRE 

Questions  1-6  relate  to  PLATO  IV  characteristics;  questions  7-14, 
to  perceived  effectiveness  of  the  Data  Link,  training  materials;  and 
questions  15-20,  to  opinions  of  CBT  in  general.  Data  for  the  objective 
questions  are  based  on  22  student  responses,  while  the  N varied  from  22 
to  28  on  the  open-ended  questions  since  more  than  one  student  response  was 
allowed  per  question. 

The  percentage  of  students  who  responded  to  the  various  alternatives 
to  questions  on  PLATO  is  shown  in  the  table  following  those  questions. 

The  percentage  of  responses  to  alternatives  to  all  other  questions  is  listed 
after  each  question. 

PLATO  IV  Ratings 

1.  Did  you  have  any  problems  or  interruptions  with  the  system  while 
proceeding  through  the  training  materials? 

2.  Did  you  have  any  problems  using  the  touch  panel? 

3.  Did  you  have  trouble  figuring  out  what  to  do  next  when  you  weren't 
told  by  the  instructor? 

4.  Did  you  have  "signing-on"  or  keyboard  problems  with  the  terminal? 

5.  Were  the  words,  drawings,  and  figures  difficult  to  see  or  read? 


6.  Did  the  delay 
disrupt  the  continuity 

in  having  to  wait 
of  the  lesson? 

for 

new  displays 

to  fill  the 

screen 

Question 

Rating  1 

2 

3 

4 

5 

6 

Never 

.23 

.05 

.64 

.14 

.27 

Seldom  .45 

.68 

.64 

.27 

.58 

.54 

Frequently  .41 

.09 

.31 

.09 

.09 

.14 

Usually  .09 

.14 

Always  .05 

.05 

.05 
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Training  Materials  Ratings 

7.  To  what  extent  do  you  feel  that  this  new  training  mode  helped 
your  understanding  of  all  basic  Data  Link  skills? 


a. 

To  a very  great  extent. 

.05 

b. 

To  a great  extent. 

.50 

c. 

To  some  extent. 

.40 

d. 

To  a little  extent. 

.05 

e. 

To  a very  little  extent. 

.00 

8.  Why  do  you  feel  the  training  was  (or 
basic  skills? 

was 

not)  effective 

in  teaching 

a. 

No  response. 

.04 

b. 

It  was  not  effective  because  it 
lacked  realism. 

.12 

c. 

It  was  effective — miscellaneous 
reasons . 

.20 

d. 

It  was  effective  because  it  allowed 
visualization  of  the  operations. 

.24 

e. 

It  was  effective  because  it  allowed 
hands-on  practice. 

.40 

9.  As 
simulations 

opposed  to  using  a workbook,  how  much  did  the  interactive  graph!* 
of  the  panel  and  tactical  plot  aid  in  learning? 

a. 

To  a very  great  extent. 

.09 

b. 

To  a great  extent. 

.91 

c. 

To  some  extent. 

.00 

d. 

To  a little  extent. 

.00 

e. 

To  a very  little  extent. 

.00 

10.  As 
performance 

an  aid  to  learning,  the  continual 
was : 

feedback  provided 

on  your 

a. 

Very  effective. 

.09 

b. 

Effective. 

.63 

c. 

Somewhat  effective. 

.23 

d. 

Ineffective. 

.05 

e. 

Very  ineffective. 

.00 
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11.  How  do  you  feel  that  this  training  capability  will  affect  your  later 
performance  in  operating  the  Data  Link  system  when  you  are  in  the  fleet? 


a.  It  will  enable  me  to  operate  the 

Data  Link  system.  .14 

b.  It  will  make  it  easier  for  me  to 
learn  the  skills  when  I use  actual 

S-3A  components.  .86 

c.  It  will  not  affect  my  performance.  .00 

d.  It  will  make  it  somewhat  difficult 

to  perform  in  the  real  situation.  .00 

e.  It  will  make  it  very  difficult  to 

perform  in  the  real  situation.  .00 


12.  What  specific  aspects  of  this  Data  Link  training  did  you  like  the  most? 


a. 

No  response. 

.12 

b. 

Miscellaneous. 

.04 

c . 

The  hands-on  practice. 

.38 

d. 

It  allows  visualization  of 
operations. 

.19 

e. 

The  similarity  to  actual  equip- 
ment and  operations. 

.15 

f . 

The  instructional  characteristics. 

.12 

What  aspects  did  you  like  the  least? 

a. 

No  response. 

.12 

b. 

Miscellaneous . 

.15 

c. 

The  size  and/or  readability  of 
the  display. 

.12 

d. 

System  or  software  problems. 

.38 

e. 

The  display  delay. 

.23 

How 

could  this  training  be  improved? 

a. 

No  response. 

.22 

b. 

Miscellaneous . 

.14 

c . 

No  changes  needed. 

.14 

d. 

Allow  more  time  to  study  it. 

.14 

e. 

Eliminate  system  or  software 
problems. 

.36 
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CBT  Ratings 


15.  In  comparison  with  other  methods  of  learning,  computer-based  training 
was : 


a. 

Very  enjoyable. 

.28 

b. 

Enjoyable. 

.67 

c. 

Neutral. 

.05 

d. 

Boring. 

.00 

e. 

Extremely  boring. 

.00 

16.  How  do  you  rate  computer-based  training  in  relation  to  other  training 
you  have  experienced? 


17. 


a. 

Outstanding. 

.09 

b. 

Above  average. 

.67 

c. 

Average. 

.19 

d. 

Fair . 

.05 

e. 

Poor , 

.00 

What 

; one  thing  do  you  like  best 

about 

computi 

a. 

Self-paced  Instruction. 

.29 

b. 

The  novelty  or  way  it  keeps 
attention. 

your 

.14 

c. 

d. 

The  instructional  characteristics 

The  capability  for  hands-on 
practice. 

.21 

.14 

e. 

The  efficiency  of  CBT. 

.11 

f. 

Interaction  with  the  system 

.11 

18.  What  one  thing  do  you  like  least  about  computer-based  training? 


a. 

No  response. 

.09 

b. 

Miscellaneous. 

.23 

c. 

Not  enough  feedback  or  instruction. 

.09 

d. 

It's  tiring  or  boring. 

.36 

e. 

The  system  or  software  problems. 

.23 
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19.  How  much  value  would  this  lesson  have  to  new  inexperienced  students 
learning  the  basic  skills  for  Data  Link  operations? 


a.  No  response.  .05 

b.  Much  value.  .59 

c.  Moderate  value.  .27 

d.  Low  value.  .09 


20.  If  you  have  any  final  comments  to  make  about  the  PLATO  IV  system, 
the  training  materials,  or  computer-based  training,  please  make  them  here. 

a.  No  response. 

b.  Miscellaneous. 


.73 

.27 
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